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5 PRINTED IMAGE WITH RELATED SOUND 

Field of the Invention 

1 0 The present invention is relevant to to a system for the provision of images with related sound. 
In particular, the present invention relates to the provision of a still image, particularly a 
printed image, together with a passage of sound attached to the still image, such that further 
information is present together with the passage of sound. 

15 

Discussion of Prior Art 

There are a number of widely used media (films, video) for provision of sound and images 
together in a concurrent continuous way, but there has been less success in the provision 

20 together of still images with associated passages of sound. This is despite the fact that this has 
been identified as a desirable combination for at least forty years. An early solution was the 
Synchrosheet technology, developed by Tokyo Denki KKK, the Tokyo Institute of 
Technology, the Dai Nippon Printing Company and the Canon Camera Company. This 
technology is described in an article by Yasushi Hoshino entitled "The Talking Book" in The 

25 Penrose Annual, Lund Humphries, London, 1959. This technology involved use of a paper 
sheet coated with magnetic film on which sound was recorded in spiral tracks using magnetic 
recording heads. A large and relatively complex reader was required to read the sound tracks. 

Alternative approaches have been adopted in subsequent work. US Patents Nos. 4,270,853 
30 and 4,270,854 both relate to instant printing cameras which also record a magnetic strip at a 
marginal portion of the film for recording sound. European Patent Application Publication 
No. 0670555 Al and US Patent No. 4983996 both relate to sound storage appended to images 
by optical means: in the former case, by a dot code, and in the latter case, by a bar code. US 
Patent No. 5128700 also teaches the use of a bar code in this context, but here the bar code 
35 does not itself contain the sound information: the bar code merely contains a reference to 
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sound stored on another medium, so that a sound reader can determine the reference from the 
bar code and play the referenced passage of sound from a dedicated sound storage medium. 
US Patent 5,276,472 also teaches use of bar codes on a photographic print as a sound storage 
medium, and further suggests features of relief (blister marks) as an alternative. US Patent 
4,905,029 discusses the idea of using "acoustic recording media" associated with photographic 
images to store related sound - alternatives involving chip storage are discussed also but these 
are stated to be "not currently practical". 

A more complex approach to this design space is found in the "Video Postcard" idea of Philips 
Electronics N.V., disclosed on World Wide Web site http://www- 
eur.philips.com/design/vof/vofsite7/postcard/index.htm. This proposal concerns a piece of 
f ilm with an embedded chip which stores a sound and video clip. The image display provision 
within the "Video Postcard" is thus specific and complex, unlike a conventional printed image. 

Proposals for realising attachment of sound to still images are also found in European Patent 
Application Publication No. EP 0827018 Al (which teaches an image player with which 
"image-audio prints" can be engaged to play sound), International Patent Application 
Publication Number WO 95/14958 and Belgian Patent Application Publication Number 
1001348A6. European Patent Application Publication Number 0851281 is relevant to 
photographic processing for media capable of supporting this type of information. 

Despite such a long standing interest in the possibility of provision of still images together with 
relevant passages of sound, no technology has been satisfactorily commercialised for this 
purpose. There is thus a need for technology which achieves provision of passages of sound 
with still images such that the sound and images together are cheap and convenient to produce, 
that playback of the sound is cheap and easy to accomplish, and so that both image and sound 
can be rendered at sufficientiy high quality to satisfy a user. It is also desirable to extend such 
technologies to provide a versatile medium capable of carrying and presenting information 
useful in a wide range of user contexts. 



5 
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Accordingly, the invention provides a system for presentation of an image and related sound, 
comprising: a printed image; an electronic storage device attached to the printed image and 
adapted to store information defining a passage of sound, and further adapted to store further 

10 information relating to the printed image; one or more devices connectable to the electronic 
storage device to enable, together or separately, the information defining a passage of sound 
to be transferred from the electronic storage device for reproduction and the further 
information to be transferred from the electronic storage device for use; and one or more 
further devices together or separately to capture the passage of sound and the further 

15 information relating to the printed image and to write the information and the further 
information into the electronic storage device. 

This arrangement allows not only for cheap, effective and robust provision of recorded sound 
in association with a printed image, but also for storage of other data together with the 
20 recorded sound. This storage of other data significantly enhances the richness of the data 
overall, and can be used to enhance the viewer experience available through this new medium 
and can give rise to new fields of use. 

This further information may include at least one form of the image (that is, the image 
25 provided as the printed image). This may be an image at equal, or even higher, resolution 
than the printed image (typically of the same resolution as the originally captured image), and 
can be used for the purpose of regenerating the printed image if the printed image becomes 
damaged, or if there is a desire for further copies. The image might instead (or as well) be 
stored at lower resolution, so that it can be provided as a thumbnail (perhaps in devices 
30 adapted to playback the sound, or to enable that sound and image will be matched correctly). 

The further information could include information obtained at capture, such as camera settings 
or the location of capture (perhaps obtained through GPS). It may also include searchable tags 
or keywords or other indexing information, or a reference to a separately held archive , or to 
35 external resources relevant to the image. It may further include arbitrary user input. A still 
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5 further possibility is the provision of security or copy control information. 

A sound reproduction device may be used as a readout device for both sound and at least some 
of this further information - it may also be used to create information, or to write it on the 
electronic storage device. 



10 



A printer (preferably that which prints the printed image) may be used to write information 
on to the electronic storage device. Advantageously, it may be used as a readout device for 
sound or for further information. 



1 5 A personal computer may be used to create further information for writing into the electronic 
storage device. 

A camera may be used to capture the image: preferably a camera is used to capture both the 
image and the sound passage. 

20 

In a further aspect, the invention also provides a method of presenting an image together with 
related sound, comprising: capturing an image; capturing a passage of sound; capturing further 
information relating to the image; printing the image to form a printed image; writing the 
passage of sound on to an electronic storage device as sound information; further writing the 
25 further information on to the electronic storage device; fixing the electronic storage device to 
the printed image; and connecting one or more devices to the electronic storage device to 
enable, together or separately, the information defining the passage of sound to be transferred 
from the electronic storage device for reproduction as sound and the further information to be 
transferred from the electronic storage device for use. 

30 

The step of fixing the electronic storage device to the printed image may, depending on the 
elements used for the purpose, be carried out before one or more of the printing of the printed 
image, the writing of the sound information, and the writing of the further information. 



35 
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5 Description of Drawings 

Embodiments of the invention are described further below, by way of example, with reference 
to the accompanying drawings, in which: 

10 Figure 1 shows the components for the process of recording, providing and playing passages 
of sound together with still images in a system suitable for the employment of embodiments 
of the present invention; 

Figures 2a and 2b show, in plan view (with detail) and side elevation respectively, the physical 
15 arrangement of an electronic storage device on to a printed image in the system of Figure 1; 

Figures 3a and 3b show in plan view and side elevation respectively the. mounting of an 
electronic storage device on to a flexible substrate to provide a device usable in the 
arrangement of Figure 2; 

20 

Figure 4 shows schematically the components required for playback of a passage of sound held 
on an electronic storage medium as a digital signal in an example of a system as shown in 
. Figure 1; 

25 Figure 5 shows schematically the components required for playback of a passage of sound held 
on an electronic storage medium as an analog signal in an example of a system as shown in 
Figure 1; 

Figure 6 shows a sound reproduction device for reproducing a passage of sound held on an 
30 electronic storage medium suitable for use in the system of Figure l;and 

Figures 7a to 7c relate to a printer particularly suitable for use in the system of Figure 1 . 



35 
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5 S pecific Description of P referred Embodiments 

Basic components in a system for providing an image and related sound are shown in Figure 
1. A camera 1 comprises conventional means for capturing a still image together with means 
for recording a passage of sound. Sound recording means are conventional: a microphone 101 
10 and an automatic gain controller 102 (in digital embodiments, such as is specifically shown in 
Figure 1, an analog/digital converter 103 and a codec 106 are also required) - where playback 
is also available, a loudspeaker 105 (and, if digital, a digital/ analog converter 104) must also 
be provided. Typically a memory 107 for storage of audio and image information will also 
be required. 



15 



20 



It should be noted that although the term "image" is used here, it does not refer merely to a 
pictorial image. It may also relate to, for example, text captured in an image format. 
Likewise, when the term "camera" is used, other devices for capturing a still image (such as 
a scanner) can be understood to fall within its scope in the present context. 



Alternatively, sound can be added separately from capture of the image (eg in later annotation) 
and thus facility for sound capture need not be provided at the camera 1 in certain versions of 
this system. However, provision of sound capture at the camera 1 is desirable as it enables 
modes of use to be employed that certain users find to be of particular value, such as the 
25 capture of ambient sound associated with the capture of a particular still image (e.g. associated 
conversation, or sounds associated with the photograph location). It also allows for annotation 
in situ on capture of the image. 

This process of image and sound capture may be entirely analog, in which case both an analog 
30 image recording medium (such as film) and an analog sound recording medium are provided 
at the camera. Either at the camera 1, or separately, the analog image is printed (or rendered 
in some other tangible form) and the passage of sound is stored in an electronic storage device 
attached to the printed image. 



35 



Alternatively, image and sound capture may be entirely digital, with both image and sound 
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5 digitized on capture and stored in an appropriate storage medium (for example on an Iomega 
"Clik" disk (for example, on an Iomega "Clik" disk ("Clik" is a trade mark of Iomega 
Corporation) or on a Sandisk "Compactflash" storage device ("Compactflash" is a trade mark of 
Sandisk). In this event, the digital data can be provided to a digital processing means (such 
as personal computer 2). This is advantageous, as it allows for easy editing of both image and 

10 sound data. The image data can then be provided from the digital processing means as a 
printed image, and the passage of sound attached thereto in a digital electronic storage 
medium, such as a flash memory. This is best achieved by means of a printer 3. In printer 
3, the image is printed in conventional maimer and a sound reproduction device with the 
passage of sound recorded thereon is attached to the image - advantageous construction of such 

15 a printer 3 is described further below. Advantageously, recording and attachment of sound 
is done in the printer itself, but this step may be carried out separately (for example, at the 
sound recording device 4, as is described further below). Alternatively, it is possible in some 
versions of the system for the image data and optionally the sound data to be provided directly 
from the camera to the printer (for example by infra red data transfer using technologies 

20 currently known for infra red printing from personal computers - such as under the IrTranP 
standard): these arrangements may require the processor in camera 1 to be provided with a 
printer driver for the appropriate printer. 

Alternatively, the processes followed may be part analog and part digital (for example, digital 
25 sound recording but analog image production, with the electronic storage medium with sound 
stored therein attached in due course to the analog- produced image). 

In each case, the result is a tangible representation of the image with an electronic storage 
medium having the passage of sound recorded thereon attached to it: this is here termed an 

30 "audioprint". To play the passage of sound, sound reproduction device 4 is employed. This 
can be connected to the electronic storage medium to enable information stored in the 
electronic storage device to be transferred to the sound reproduction device, the sound 
reproduction device 4 containing means to convert the information received from the electronic 
storage device into the passage of sound. However, sound reproduction device 4 is adapted 

35 to be detachable or otherwise remote from the printed image when no connection between the 
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5 electronic storage device and the sound reproduction device is required. 

As the person skilled in the art will appreciate, the system of Figure 1 could be expanded or 
modified by use of different components with different or expanded functionality, or by 
appropriate communication mechanisms between the components. Any component with 

10 appropriate functionality for capture, writing of sound or images, or for playback of sound, could 
in potential be used, as could any component type that could be effectively modified for such a 
purpose or to include such a purpose along with its existing functionality. If appropriate 
communication mechanisms were present, any appropriate communication type or path could 
be used between appropriate system components. For example, using a protocol such as JetSend 

15 (developed by the present applicant and described at http://www.jetsend.hp.com/JSHome.html) 
direct communication between a camera and a printer, or any other pair of JetSend enabled 
appliances, to transfer image or sound content or both in an appropriate negotiated form would 
be possible. Other possibilities could include using facsimile as a way to transfer information 
for printing - essentially any appropriate communication protocol could be used. 

20 

Not shown in Figure 1 is the tangible image and the electronic storage medium. This is 
illustrated in Figures 2a and 2b. Figure 2a shows an image 1 1 in a tangible form: in this case 
a printed image. Attached to the image 11 is an electronic storage device 12. The electronic 
storage device comprises in this case a die 16 containing a memory (which may be for example 
25 a flash memory, or another form of EEPROM or PROM) in which the passage of sound is 
recorded, the die 16 being connected by tracks to connectors 15. The die 16, the tracks and 
the connectors 15 are all mounted on a flexible substrate 14, the flexible substrate being a 
membrane, advantageously of a plastics material. 



30 Figure 2b shows one particularly advantageous arrangement for reproducible location of the 
electronic storage device 12 with respect to the tangible image 11. This is for the image 11 
to be printed on paper 17 provided with a recess 13. A particularly appropriate form for the 
recess is as a slot at the edge of the paper sheet with a depth roughly equal to the thickness of 
the storage device. This leaves the resulting image 11 in a form which is particularly easy to 

35 handle (no more difficult than a conventional photographic print, or a sheet of paper). 
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5 

The construction of a particularly suitable electronic storage device, here termed an audiotab, 
is shown in Figure 3a. Die 16 contains a memory device - advantageously a flash memory or 
other non-volatile EEPROM, though another form of PROM could provide a suitable 
alternative. As shown in Figure 3b, die 16 is fixed to the flexible substrate 14 by a number 

10 of solder bumps 18 to establish electrical connection between the die 16 and the conducting 
tracks on the substrate 14, and an insulating material is provided as a fill 19 to bind the die 16 
into a common structure with the substrate 14. As previously indicated, it is desirable for the 
substrate 14 to be flexible: it is desirable that the substrate is at least of comparable flexibility 
to the printed image 11, otherwise the chance of the electronic storage device becoming 

15 detached increases significantly and the printed image will become more difficult to handle. 

In the embodiment shown, connections to the audiotab are made by contact with a device 
(generally the sound reproduction device) abutting the edge of the printed image 11 in some 
way. Other forms of connection are quite possible. For example, connection could be made 

20 by surface contact, rather than by an edge connection of this type. Alternatively, signals and 
power could be transmitted without direct contact (signals by a variety of means, power 
typically by induction). It should also be noted that although connections to the audiotab for 
both signal and power are provided together (in preferred embodiments the audiotab will have 
no source of power itself, but will draw power from a device accessing it), this is not a 

25 necessary feature of the invention, and the two could be provided through different connectors, 
for example. 

Also possible are different types of audiotab construction. For example, it may be possible 
to print appropriate circuitry directly on to an appropriate substrate material, removing the 
30 need for a chip (and hence for the mounting of the chip on to the substrate). 

Each element of a preferred version of the system here described will now be discussed in 
greater detail, both with regard to its construction and to its function. Information flow 
between elements of the system is also discussed. There are a number of particularly 
35 advantageous design options associated with the different elements of the system, some of 
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these options being appropriate to certain overall uses of the system or components of the 
system, and some to others. These options are described with regard to the system element 
to which they relate, and where appropriate will be further described in the context of modified 
systems discussed further below. The role of the present invention in the context of such a 
system will be described. 

The camera 1 is adapted in different embodiments of the invention to capture images in various 
different, but conventional, ways. In preferred arrangements, the image will be captured and 
stored as a digital representation as in a conventional digital camera. However, arrangements 
are also available where the image is prepared by an analog route - in which case the image 
may be recorded on to conventional photographic film, or produced as an instant printed image 
(Land camera). In such analog arrangements, sound is handled separately, and image and 
sound information take different processing routes until the tangible image and the electronic 
storage device are assembled together. 

Where camera 1 differs from a conventional camera is that it also contains means for recording 
a passage of sound. This feature is known from and discussed in various of the prior art 
documents mentioned earlier in the application. Essentially all that is required is that a 
microphone 101 and a sound recording apparatus are provided, with certain basic controls and 
displays: means to start and stop recording and to associate a specific recording with a specific 
image, and preferably means to display when a given recording is in progress and means for 
sound playback. Optional features are that recording could be synchronised with taking of a 
picture (starting or stopping at the point of image capture, or with image capture occurring at 
a predefined point during sound capture) - other conventional sound recording features (stereo, 
noise reduction etc.) may also be provided. Further features in the user interface may also be 
provided so as to allow the user to record additional information for storage together with the 
sound in the audiotab. Examples of such further information are discussed in more detail 
below - however, in this context they may include user-entered characters, a time or position 
of recordal (in the latter case, possibly a value obtained through a GPS receiver), or details 
of the camera settings (exposure time, focussing details etc.). 
Advantageously, sound is recorded digitally and held in an appropriate storage medium. In 
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5 preferred arrangements, both sound and images are recorded on the same digital storage 
medium (for such digital storage an Iomega "Clik" disk would be an appropriate choice), but 
in alternative arrangements sound and image can be handled through separate media (for 
example, if the image is recorded on film). Alternatively, the electronic storage medium for 
attachment to the tangible image could be recorded directly at the camera: this would be 
10 appropriate for a Land camera arrangement, for example. Discussion of recording on to an 
appropriate electronic storage medium will be discussed further at a later point. 

In addition to recording capability, the camera 1 may also have the function of a sound 
playback device capable of reproducing sound from the information stored in the electronic 
15 storage medium attached to the tangible image. Again, the necessary functionality is indicated 
below in the discussion of sound playback device 4. 

For digital image or sound data, the next stage is signal processing by a signal processing 
device, which in appropriate embodiments will be a personal computer 2. Again, substantially 

20 any conventional image processing software may be used for editing or otherwise processing 
a digital image: likewise, conventional software is available for editing or otherwise changing 
the associated passage of sound. Examples of appropriate conventional software are Creative 
Wave Studio, Studio M (a product of Gold Disk, Inc.) and Adobe Premiere. 
In the system of Figure 1, the processed digital data in personal computer 2 can then be 

25 handled independently: the image can be sent to a printer 3, and the sound to a device for 
recording sound on to the chosen electronic storage medium, with consequent assembly 
(manual or otherwise) of the printed image and the electronic storage medium together. The 
audiotab recording could then take place in the sound reproduction device (discussed further 
below), by an attachment or peripheral to the personal computer 2, or by a separate storage 

30 appliance with the capacity to store sound clips and record audiotabs with the stored clips. 
Such a storage appliance could also comprise means to archive sound clips (and any further 
recorded information, such as images, and to provide means to recover on request sound clips 
and any related information from the storage appliance. 



35 



Advantageously, the printer 3 has both the function of printing the image and of transferring 
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the passage of sound on to the electronic storage medium. The personal computer 2 is 
adapted to send image information to the printer for printing in conventional manner: however 
a way is also provided for the associated file containing the passage of sound to be provided 
too (this is easily accomplished by use of a file format with an extension for sound)). Before 
or after the image is printed, the sound file is also transmitted to the printer. Means are 
provided at the printer for recording the sound file on to the electronic storage medium. 

The mechanically simplest solution, shown in Figure 7a, is to provide a discrete interface 801 
at the printer 802 into which a blank sound storage device 803 could be slotted, the storage 
device then being removed after recording and fixed by the user to the printed image 804 (a 
separate fixing means could be provided at the printer, such as an embossing head 805, so that 
the printed image 804 and the sound storage device 803 could be fixed together by a user after 
each was, separately, produced). Alternatively, as shown in Figure 7b, means could be 
provided whereby the printer 802 could be loaded with a supply of blank sound storage 
devices, a single sound storage device being recorded with a passage of sound on printing of 
an associated printed image. In the version shown in Figure 7b, the blank devices are 
provided in the form of cartridges 806, but could also be provided in the form of a bandolier, 
for example. It is desirable to be able to pre-load the printer with a number of blank devices, 
as this minimises user intervention, particularly in arrangements which allow for batch 
downloading and printing of a number of clips . 

A particularly preferred form of this arrangement includes automatic paper handling to hold 
the printed image and to fix the associated sound storage device on to the printed image: the 
image and attached sound are thus output from the printer 3 without manual intervention from 
the user. In a particularly effective solution, a specific path is provided for sound storage 
device substrates in the printer mechanism. This path could be designed for single substrates, 
but a more efficient mechanism would be provided for substrates provided in cassette or 
bandolier form. The bandolier solution is effective for self-adhesive substrates, as it is 
straightforward then to use a substrate path in which the tape is peeled away, shortly prior to 
substrate attachment, to reveal the self-adhesive surface. The printer path could provide for 
indentation of the print medium before die attachment (alternatively, pre-indented paper for 
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5 accommodation of audiotabs could be pre-loaded into the printer). The printer and substrate 
path then run together to an attachment point, at which both the print medium and audiotab 
substrate are held in position, then fixed together by appropriate mechanical action (e.g. 
pressed together under solenoid action). Appropriate process steps are shown in Figure 7c. 

10 In printers which did not support such automatic paper handling, there would need to be also 
some mechanism for reconciling a printed audiotab with a printed image (particularly if 
multiple audiotabs can be recorded). This could be, for example, a numbered symbol printed 
on the page which corresponded to a numbering of the audiotab. Another approach which 
could be used if a version of the image was electronically stored on the audiotab (see below) 

15 would be to display a thumbnail of the image relating to an audiotab to be dispensed on a 
printer display. *. 

Modifications required to existing printer structures are not so fundamental that they would 
pose particular difficulty to a person skilled in state of the art printer construction. 
20 Modification to existing communication with printers to support additional functions also poses 
no fundamental difficulty - standard languages used for the purpose, such as PCL, could 
readily be extended with special commands recognised by the printer (such commands could 
for example be placed in a comments field, so that they will be ignored by printers unable to 
produce audiotabs). 

25 

Further developments to printers of this type can be provided. If the printer 3 possesses not 
only a means for writing audiotabs, but also a means for reading audiotabs (ie it includes the 
functionality of reader device 4 in this respect, perhaps without need for audio playback), and 
the printer 3 is adapted to act as a photocopier, then the printer 3 as a whole may be adapted 

30 to act as an audioprint photocopier: both a printed image and an attached audiotab may be 
copied (probably in a separate copying process, but possibly at the same time if the overall 
printer design allows for the various printer components to be located appropriately) and a new 
print and associated audiotab provided. A further possibility is for the printer to record 
audiotabs alone (without images) if required - possibly from sound recorded at a reader device 

35 4, or if appropriate circuitry is provided, at the printer 3 itself. The printer 3 then may permit 
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stand alone audio capture, recording into an audiotab, and replay, all to be carried out. 

The sound reproduction device 4 will now be described with reference to Figure 6. Sound 
reproduction device 4 has a slot 24 adapted to receive connectors 15 of the electronic storage 
device 12. This allows access to the memory in the electronic storage device, and hence to 
the passage of sound, by the sound reproduction device 4. The reproduction device has user 
operable switches: in the embodiment shown in Figure 6, there is a play /stop button 25 and 
a rewind button 26. Sound reproduction device 4 also comprises a loudspeaker. Further 
functional features normal in sound reproduction devices can also be provided: for example, 
a headphone/earpiece connection, a fast forward button and a volume control. Capability can 
also be provided for recording passages of sound on to the electronic storage device at the 
sound reproduction device 4. The passage of sound recorded on to the electronic storage 
device can then be modified or replaced at the sound reproduction device 4. The circuitry 
necessary for sound reproduction device 4 is discussed further below together with associated 
features of the audiotab. 



In different arrangements, sound is provided from the camera 1 directly to the sound 
reproduction device 4 (perhaps through a temporary storage on a different storage medium, 
or simply from a standard audio output), with the initial recording of sound on to the electronic 
storage device taking place at the sound reproduction device, either before or after the fixing 
of the electronic storage device to the printed image. Alternatively, this direct connection can 
be from the personal computer 2 to the sound reproduction device 4, with recording of the 
audiotab at the sound reproduction device 4. Sound can be processed in the personal computer 
2 as indicated above with conventional software, and then sent to the sound reproduction 
device 4 for subsequent audiotab recording. Another variant that effectively combined features 
is for camera 1 and sound reproduction device 4 to be the same object: this is a practical 
approach as sound recording and playback capability are also present in the camera. Likewise, 
alternative arrangements employ personal computer 2, with appropriate peripheral circuitry, 
as sound reproduction device 4. 



The circuitry of the electronic storage device and the sound reproduction device will now be 
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5 described. Other circuitry, and programs, are essentially conventional and the person skilled 
in the art will be well aware of the choices that are available 

A digital solution for sound reproduction from sound stored on the electronic storage device 
is shown in Figure 4. Die 16 of the electronic storage device 12 contains a non-volatile 

10 memory. In a preferred design choice, electronic storage device 12 is a CMOS device having 
an on-chip oscillator, a high density flash memory storage array, a serial interface, a write 
buffer and an address decoder. The bondout pitch of die 16 is expanded by the tracks on the 
substrate 14 to provide a connector 15 which can readily interface with an appropriately 
matched connector 51 of the sound reproduction device 4. At this connector, a separate 

15 connection may be provided for every input and output required (signal, power) or 
alternatively these may be combined with appropriate conventional additional circuitry (for 
example, the signal may be provided by modulation of the power connection, in which case t yz 

a modulator/demodulator circuit is also required). The arrangement shown in Figure 4 shows ■ in- 

direct access to the memory in die 16 by the sound reproduction device and direct playing of 

20 the sound recording device: an alternative solution is for appropriate means to be provided to 

first download some or all of the information stored in the memory on die 16 to a separate $ 
memory in the sound reproduction device 4 for fast access by the reproduction device. As the . . * 

sound is recorded in digital form, it needs to pass through digital to analog converter 52 and 
normally an amplifier 53 and appropriate filtration and gain stages 54 before rendering as 

25 sound through loudspeaker 55 (or alternatively provided on headphone/line output 56). 
Functions such as play, stop, rewind and fast forward are provided by conventional circuitry 
from manual switches 57. Recording at the sound reproduction device requires additionally 
microphone input 59, an analog to digital converter 58 and means to write to the memory in 
die 16. A power source for the sound reproduction device 4 is also needed, though not shown 

30 here (this may be a battery, for example). No separate power source is required for the 
electronic storage device, as it does not need to draw power except when connected to the 
sound reproduction device, which provides the power. Sound compression and decompression 
can also be used to maximise storage efficiency: a separate stage for decompression of stored 
data can be provided before digital to analog conversion (and likewise after analog to digital 

35 conversion on recording) with conventional technology. 
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An analog sound storage solution is shown in Figure 5. The arrangement of Figure 5 is 
substantially similar to that of Figure 4, and equivalent components are given the same 
reference numbers and are not described further here. In the analog case, no digital to analog 
converter 52 (or analog to digital converter on recording) is required: only a buffer (not 
shown) is required before amplifer 53. An appropriate analog storage technology is the 
ChipCorder technology of Information Storage Devices, Inc. (ISD), which provides a true 
quantised multilevel representation of the sample per cell. In this case the electronic storage 
device 12 is preferably again a CMOS device with an on-chip oscillator and high density 
multilevel EEPROM (such as ChipCorder, discussed above) storage array; an antialiasing filter 
and a smoothing filter will also be required in the overall circuitry. Connection choices in 
supply of power and signals can be as for the digital case. 

In embodiments of the invention, further information beyond simply the passage of sound may 
be recorded in the memory on electronic storage device 12. Clearly, this is particularly 
appropriate where sound is stored digitally on electronic storage device 12. Such information 
could be entered and recorded at camera 1 (this may include information such as the time of 
recording) or may contain extensive annotation or other information provided at personal 
computer 2, sound reproduction device 4, or elsewhere: such information may be provided as 
a text file, or in an appropriate file format. Examples of particular additional information, and 
25 the consequent advantages, will now be discussed. 

It is particularly advantageous for the memory of the electronic storage device 12 to be 
recorded with a representation of the image to which it is (or is to be) attached - this enables 
the image to be recreated if, for example, the printed image is damaged. This could be, for 
30 example, the image represented as a GIF file. This image could be provided as a full 
representation of the original image (to allow recreation as discussed above) but could also be 
provided as a thumbnail for use as a low cost method of identifying the sound clip. Provision 
of such additional information makes it strongly advantageous for sound reproduction device 
4 to also contain a display (not shown). This display may be adapted to show some or all of 
35 the additional information recorded in the memory, possibly including a representation of the 
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5 image itself. 

Additional information associated with the actual capture of the image is most easily provided 
at the camera itself. Date and time information and camera settings are typically available 
electronically within a conventional digital camera, so it is therefore straightforward for these 
10 to be recorded together with the sound clip. Location could be recorded either by user input, 
or conceivably by means of a GPS receiver or similar location sensing means within the 
camera itself. Audio settings could be recorded in a similar manner to camera settings. User 
annotation could also be provided at this point. 

15 Other information types that could be provided would more typically be added in processing 
of an image at personal computer 2. A particularly useful indication is a reference to an 
archive (for example of an image of full quality, if only a thumbnail or no image is recorded 
on the audiotab - such an archive reference might be in the form of a web URL, a personal 
computer directory, or a flash card number. A further possibility would be the provision of 

20 images related to the printed image, as well as (or instead of) the image of the printed image 
itself. The further information could include a video clip, perhaps a video clip to which the 
image of the printed image acted as a reference image. Other information added could be 
information to assist in searching, such as classification of the image by type (for example, 
identifiers for the captured scene) or of the audio by type (for example, speech or ambient 

25 sound), or simply to identify relevant or associated material (again, a web URL on the public 
internet for a site associated with the captured scene). Other information typically added at 
the personal computer point could be related to security, or other forms of control (for 
copyright licensing purposes, for example) - such information could be a total number of 
copies of the data allowed, or text information indicating the copyright owner. The sound 

30 reproduction device 4 would again be an appropriate place to add information, particularly 
annotation by the viewer of the image. 
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CLAIMS 

1. System for presentation of an image and related sound, comprising: 
a printed image; 

an electronic storage device attached to the printed image and adapted to store 
information defining a passage of sound, and further adapted to store fixrther 
information relating to the printed image; 

one or more devices connectable to the electronic storage device to enable, together or 
separately, the information defining a passage of sound to be transferred from the 
electronic storage device for reproduction and the further information to be transferred 
from the electronic storage device for use; and 

one or more further devices together or separately to capture the passage of sound and 
the further information relating to the printed image and to write the information and 
the further information into the electronic storage device. 

2. A system as claimed in claim 1 , wherein said further information includes at least one 
form of the image provided as the printed image. 

3 . A system as claimed in claim 2, wherein the form of the image provided has resolution 
substantially equivalent to the printed image. 

4 A system as claimed in claim 2, wherein the form of the image provided has resolution 
substantially lower than the printed image. 

5 . A system as claimed in any preceding claim, wherein said further information includes 
a record of camera settings in capture of the image provided as the printed image. 
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6. A system as claimed in any preceding claim, wherein said further information includes 
location identification relating to the location at which the image provided as the 
printed image was captured. 

7. A system as claimed in claim 6, wherein said location information is obtained from a 
global positioning system device in the image capture device. 

8. A system as claimed in any preceding claim, wherein said further information includes 
descriptors relating to the image provided as the printed image for use in searching or 
indexing of the image. 

9. A system as claimed in any preceding claim, wherein said further information includes 
a description of the image provided as the printed image entered by a user. 

10. A system as claimed in any preceding claim, wherein said further information includes 
a reference to remotely stored data related to the image provided as the printed image. 

11. A system as claimed in any preceding claim, wherein said further information includes 
a reference to a remotely stored archive including the image provided as the printed 
image. 

12. A system as claimed in any preceding claim, wherein the further information includes 
security or copy control information. 

13. A system as claimed in any preceding claim, wherein the electronic storage device is 
fixedly attached to the printed image. 

14. A system as claimed in any preceding claim, wherein one of the one or more devices 
is a sound reproduction device. 
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15. A system as claimed in claim 14, wherein said sound reproduction device is also one 
of the one or more further devices. 



16. A system as claimed in any preceding claim, wherein one of the one or more further 
devices is a printer. 

17. A system as claimed in claim 15, wherein the printer is the printer which produces the 
printed image. 

18. A system as claimed in claim 16 or claim 17, wherein the printer is adapted to write 
the information and the further information to the electronic storage device, but is not 
adapted to capture the passage of sound. 

19. A system as claimed in any of claims 1 6 to 18 , wherein the printer is also one of said 
one or more devices. 

20. A system as claimed in claim 19, wherein the printer is adapted to reproduce the 
passage of sound. 

21. A system as claimed in claim 19 or claim 20, where the printer is adapted to display 
at least one element of the further information. 



22. A system as claimed in claim 21 where dependent on claim 2, where the printer is 
adapted to display the version of the image provided as the printed image contained in 
the further information. 

23. A system as claimed in any of claims 16 to 22, wherein the printer is adapted for the 
capture or user input of at least one element of the further information. 

24. A system as claimed in any preceding claim, wherein one of the one or more further 
devices is a personal computer. 
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25. A system as claimed in claim 24, wherein the personal computer is used to provide or 
create at least one element of the further information. 

26. A system as claimed in any preceding claim, wherein one of the one or more further 
devices is a camera used for capture of the image provided as the printed image. 

27. A system as claimed in claim 26, wherein the camera is also used for the capture of the 
passage of sound. 

28. A system as claimed in claim 26 or claim 27, wherein the camera is also used for the 
capture of at least one element of the further information. 

29. A system as claimed in claim 28, wherein said at least one element relates to aspects 
of the image capture and is provided automatically by the camera. 

30. A system as claimed in claim 28, wherein said at least one element is provided by user 
input. 

31. A method of presenting an image together with related sound, comprising: 
capturing an image; 

capturing a passage of sound; 

capturing further information relating to the image; 

printing the image to form a printed image; 

writing the passage of sound on to an electronic storage device as sound information; 
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further writing the further information on to the electronic storage device; 
fixing the electronic storage device to the printed image; and 

connecting one or more devices to the electronic storage device to enable, together or 
separately, the information defining the passage of sound to be transferred from the 
electronic storage device for reproduction as sound and the further information to be 
transferred from the electronic storage device for use. 

32. A method as claimed in claim 31, wherein the further information includes a form of 
the image. 

33. A method as claimed in claim 31 or claim 32, wherein the image and the passage of 
sound are both captured in a camera. 

34. A method as claimed in any of claims 31 to 33, wherein the fixing of the electronic 
storage device to the printed image occurs before one or more of the printing of the 
printed image, the writing of the sound information, and the writing of the further 
information. 

35. A method as claimed in any one of claims 31 to 34, wherein one of the one or more 
devices is a sound reproduction device. 

36. A method as claimed in claim 35 , wherein the writing of at least a part of the further 
information is carried out by the sound reproduction device. 



37. 



A method as claimed in any one of claims 31 to 36, wherein the writing of the sound 
information and the writing of at least a part of the further information is carried out 
by a printer used to print the printed image. 
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Figure 2b 
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Figure 3b 
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Figure 6 



WO 00/03298 



PCT/GB99/02190 




Figure 7b 
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